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TITLE OF THE INVENTION 
INFORMATION PRESENTING APPARATUS, OPERATION PROCESSING 
METHOD THEREFOR, STORAGE MEDIUM STORING PROGRAM FOR 
EXECUTING OPERATION PROCESSING 

5 

FIELD OF THE INVENTION 

The present invention relates to an information - 
presenting apparatus for combining pieces of information 
such as an on-the-spot picture taken by a camera, map, and 
10 text document, and presenting the resultant information to 
a user, an information presenting method, and a 
computer-readable storage medium used for the apparatus and 
method. 

15 BACKGROUND OF THE INVENTION 

Conventionally, information presenting apparatuses 
are available, which acquire and display information such 
as pictures at camera angles designated by users operating 
cameras in remote places through networks such as the 

20 Internet upon operating devices at the user side. Some 
apparatus is designed to display a map of an area around 
the installation position of a camera on the apparatus, 
graphically show the camera angle, and operate the camera 
angle on the map, as well as operating the camera angle by 

25 directly designating the panning value, tilt value, and 
zoom value of the camera. 
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With an apparatus of this type, a user can virtually 
experience a remote place. In many cases, the user wants 
to see a specific object and know information about an 
object image-sensed by a camera. 
5 Information about an object is provided by the WWW 

on the Internet and the like, and a user can browse the 
information with an apparatus having a WWW browser. 
However, the above conventional information presenting 
apparatus is not linked to such an information browsing 
10 apparatus. For this reason, it takes much time and labor 
for the user to see information to be browsed and operate 
the camera at the same time to see an object described with 
the browsed information or search the WWW or the like for 
information about an image-sensed object. 

15 

SUMMARY OF THE INVENTION 
The present invention has been made in consideration 
of the above problems, and has as its object to 
automatically provide information about an object 
20 image-sensed by a camera and automatically image-sense and 
present an object described by browsed information while 
a user is browsing the information. 

In order to achieve at least one of the objects 
described above, according to an embodiment of the present 
25 invention, there is provided an information presenting 
system comprising: scope holding means for holding 



a scope indicating an object to which a user is currently 
giving attention in accordance with at least one of a 
picture sensed by a camera, a map indicating a position of 
the camera, and a text document; picture operating means 
5 for changing an image-sensing range of the camera; first 
instruction means for instructing the camera to switch 
display pictures when the image-sensing range of the camera 
is changed by the camera operating means or the scope held 
in the scope holding means is changed, and instructing the 

10 scope holding means to hold a new scope corresponding to 
a new image-sensing range when the image-sensing range of 
the camera is changed by the picture display means; picture 
display means for displaying a picture within the 
image-sensing range of the camera which is designated by 

15 an instruction from the first instruction means; map 

operating means for changing the scope in the scope holding 
means by designating an object on the map by giving an 
instruction; second instruction means for updating display 
of a symbol indicating the image-sensing range of the camera 

20 which is displayed on the map when the scope is changed by 
the map operating means or the scope held in the scope 
holding means is changed, and instructing the scope holding 
means to hold a new scope when the scope is changed by the 
map operating means; map display means for displaying a map 

25 designated by an instruction from the second instruction 
means; document operating means for changing a display 



document by operating a text document; third instruction 
means for giving an instruction to switch documents when 
the display document is changed by the document operating 
means or the scope held in the scope holding means is changed, 
5 and instructing the scope holding means to hold a new scope 
when the display document is changed by the document 
operating means; and document display means for - 
displaying the document designated by an instruction from 
the third instruction means. 

10 According to another aspect of the present invention, 

there is provided an information presenting method 
comprising: the scope holding step of holding a scope 
indicating an object to which a user is currently giving 
attention in accordance with at least one of a picture 

15 sensed by a camera, a map indicating a position of the camera, 
and a text document; the picture operating step of changing 
an image-sensing range of the camera; the first instruction 
step of instructing the camera to switch display pictures 
when the image-sensing range of the camera is changed in 

20 the camera operating step or the scope held in the scope 
holding step is changed, and giving an instruction to hold 
a new scope corresponding to a new image-sensing range in 
the scope holding step when the image-sensing range of the 
camera is changed in the picture display step; the picture 

25 display step of displaying a picture within the 

image-sensing range of the camera which is designated by 



an instruction in the first instruction step; the map 
operating step of changing the scope in the scope holding 
step by designating an object on the map by giving an 
instruction; the second instruction step of updating 
5 display of a symbol indicating the image-sensing range of 
the camera which is displayed on the map when the scope is 
changed in the map operating step or the scope held in the 
scope holding step is changed, and giving an instruction 
to hold a new scope in the scope holding step when the scope 

10 is changed in the map operating step; the map display step 
of displaying a map designated by an instruction from in 
the second instruction step; the document operating step 
of changing a display document by operating a text document; 
the third instruction step of giving an instruction to 

15 switch documents when the display document is changed in 
the document operating step or the scope held in the scope 
holding step is changed, and giving an instruction to hold 
a new scope in the scope holding step when the display 
document is changed in the document operating step; and the 

20 document display step of displaying the document designated 
by an instruction in the third instruction step. 

According to still another aspect of the present 
invention, there is provided a computer-readable medium 
storing a program comprising program codes of: the scope 

25 holding step of holding a scope indicating an object to 
which a user is currently giving attention in accordance 



with at least one of a picture sensed by a camera, a map 
indicating a position of the camera, and a text document ; 
the picture operating step of changing an image-sensing 
range of the camera; the first instruction step of 
5 instructing the camera to switch display pictures when the 
image-sensing range of the camera is changed in the camera 
operating step or the scope held in the scope holding step 
is changed, and giving an instruction to hold a new scope 
corresponding to a new image-sensing range in the scope 

10 holding step when the image-sensing range of the camera is 
changed in the picture display step; the picture display 
step of displaying a picture within the image-sensing range 
of the camera which is designated by an instruction in the 
first instruction step; the map operating step of changing 

15 the scope in the scope holding step by instructing an object 
on the map by giving an instruction; the second instruction 
step of updating display of a symbol indicating the 
image-sensing range of the camera which is displayed on the 
map when the scope is changed in the map operating step or 

20 the scope held in the scope holding step is changed, and 
giving an instruction to hold a new scope in the scope 
holding step when the scope is changed in the map operating 
step; the map display step of displaying a map designated 
by an instruction in the second instruction step; the 

25 document operating step of changing a display document by 
operating a text document; the third instruction step of 



giving an instruction to switch documents when the display 
document is changed in the document operating step or the 
scope held in the scope holding step is changed, and giving 
an instruction to hold a new scope in the scope holding step 
when the display document is changed in the document 
operating step; and the document display step of displaying 
the document designated by an instruction in the third 
instruction step. 

Other features and advantages of the present 
invention will be apparent from the following description 
taken in conjunction with the accompanying drawings, in 
which like reference characters designate the same or 
similar parts throughout the figures thereof. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a block diagram showing the basic 
arrangement of an information presenting apparatus 
according to the first embodiment; 

Fig. 2 is a block diagram showing a practical 
arrangement of the information presenting apparatus 
according to the first embodiment; 

Fig. 3 is a view showing a display example according 
to the first embodiment; 

Fig. 4 is a flow chart showing information display 
processing according to the first embodiment; 

Fig. 5 is a block diagram showing the basic 



arrangement of an information presenting apparatus 
according to the second embodiment ; 

Fig. 6 is a view showing a display example according 
to the second embodiment ; 
5 Fig. 7 is a block diagram showing the basic 

arrangement of an information presenting apparatus 
according to the third embodiment; 

Fig. 8 is a view showing the arrangement of an 
image-sensing condition holding unit according to the third 
10 embodiment; 

Fig. 9 is a view showing a display example according 
to the third embodiment; 

Fig. 10 is a flow chart showing information display 
processing according to the third embodiment; 
15 Fig. 11 is a block diagram showing the basic 

arrangement of an information presenting apparatus 
according to the fourth embodiment; 

Fig. 12 is a view showing the arrangement of an 
image-sensing condition holding unit according to the 
20 fourth embodiment; 

Fig. 13 is block diagram showing the basic 
arrangement of a camera controller according to the fifth 
embodiment ; 

Fig. 14 is a block diagram showing a practical 
25 arrangement of the camera controller according to the fifth 
embodiment ; 



Fig. 15 is a view showing a display example according 

to the fifth embodiment; 

Fig. 16 is a view showing the arrangement of an area 

data holding unit according to the fifth embodiment; 
5 Fig. 17 is a view showing the arrangement of a 

conversion parameter holding unit according to the fifth 

embodiments- 
Fig. 18 is a view showing the arrangement of an 

image-sensing condition holding unit according to the fifth 
10 embodiment ; 

Fig. 19 is a flow chart showing processing according 

to the fifth embodiment; 

Fig. 20 is a block diagram showing the basic 

arrangement of a camera controller according to the sixth 
15 embodiment ; 

Fig. 21 is a block diagram showing the basic 

arrangement of a camera controller according to the seventh 

embodiment ; 

Fig. 22 is a block diagram showing a practical 
20 arrangement of the camera controller according to the 
seventh embodiment; 

Fig. 23 is a view showing a display example on a 
display according to the seventh embodiment; 

Fig. 24 is a view showing a display example on a 
25 display according to the first embodiment; 

Fig. 25 is a view showing another display example on 



the display according to the first embodiment; 

Fig. 26 is a view showing still another display 
example on the display according to the first embodiment; 
and 

5 Fig. 27 is a view showing still another display 

example on the display according to the first embodiment. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
The embodiments of the present invention will be 
10 described below with reference to the accompanying 
drawings . 
(First Embodiment) 

Fig. 1 is a block diagram showing the basic 
arrangement of an information presenting apparatus 
15 according to the first embodiment of the present invention. 

Referring to Fig. 1, a scope holding unit 101 holds 
an object (to be referred to as a scope hereinafter) to which 
a user currently gives attention. Upon receiving a 
notification indicating a change of scope from a map 
20 switching unit 106 or document switching unit 109, the scope 
holding unit 101 transfers the information indicating the 
change of scope to the picture switching unit 103, map 
switching unit 106, and document switching unit 109. 

A picture operating unit 102 changes an image-sensing 
25 range by changing the direction or zoom value of a camera 
(not shown) installed in a remote place. A target located 
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in the new image-sensing range becomes a new scope. The 
picture switching unit 103 gives an instruction to switch 
pictures when the image-sensing range is changed by the 
picture operating unit 102 or a notification indicating a 
5 change of scope is received from the scope holding unit 101 . 
When the image-sensing range is changed by the picture 
operating unit 102, the picture switching unit 103 
transfers information indicating the change of scope to the 
scope holding unit 101. 
10 A picture display unit 104 acquires a picture in the 

image-sensing range designated by an instruction from the 
picture switching unit 103 from an external camera, and 
displays it. 

A map operating unit 105 changes a scope by operation 
15 on a map. The scope selected by this operation becomes a 
new scope. The map switching unit 106 gives an instruction 
to switch maps when several scopes are selected by the map 
operating unit 105 or a notification indicating a change 
of scope is received from the scope holding unit 101. When 
20 a scope is changed by the map operating unit 105, the map 
switching unit 106 transfers information indicating the 
change of scope to the scope holding unit 101 . A map display 
unit 107 displays the map designated by an instruction from 
the map switching unit 106. 
25 A document operating unit 108 changes a displayed 

document by operating a hypertext document. A scope 



changes in accordance with the displayed document 
designated by the document operating unit 108. The 
document switching unit 109 gives an instruction to switch 
documents when a displayed document is changed by the 
5 document operating unit 108 or a notification indicating 
a change of scope is received from the scope holding unit 
101. When the scope is changed by the document operating 
unit 108, the document switching unit 109 transfers 
information indicating the change of scope to the scope 

10 holding unit 101. The document display unit 110 displays 
the document designated by an instruction from the document 
switching unit 109. 

Fig. 2 is a block diagram showing a practical 
arrangement of the information presenting apparatus 

15 according to this embodiment. 

Referring to Fig. 2, a CPU 201 controls the overall 
information presenting apparatus and operates in 
accordance with a program implementing the following 
procedure. A RAM 202 provides the scope holding unit 101 

20 and a storage area required for processing operation of the 
above program. A disk unit 203 holds a program for 
implementing the procedure to be described later. A 
display 204 displays the picture display unit 104, map 
display unit 107, and document display unit 110. A mouse 

25 205 is used to input operation performed by the user, and 
serves as part of the picture operating unit 102, map 
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operating unit 105, and document operating unit 108. A 
network interface 206 acquires a picture in a designated 
image-sensing range from the external camera through the 
network. A bus 207 is used to exchange data within the 
5 information presenting apparatus. 

Fig. 3 shows various pieces of information displayed 
on the display 204 of the information presenting apparatus. 

Referring to Fig. 3, the picture display unit 104 
outputs a picture 301. The picture 301 is displayed at an 

10 angle covering all objects corresponding to the scope held 
in the scope holding unit 101. The map display unit 107 
outputs a map 302, on which the angle of the camera, which 
covers all the objects corresponding to the scope held in 
the scope holding unit 101, is graphically displayed as 

15 indicated by a symbol 314 in Fig. 3. A document display 
unit 110 outputs a document 303 . The document 303 describes 
an object corresponding to the scope held in the scope 
holding unit 101. 

When the user moves slide bars 311, 312, and 313 near 

20 the picture 301 with the mouse, the information presenting 
apparatus of this embodiment can inform the picture 
operating unit 102 of the operation for the picture. The 
user can pan the camera by horizontally moving the slide 
bar 311; can tilt the camera by vertically moving the slide 

25 bar 312; and can change the zoom magnification of the camera 
by vertically moving the slide bar 313. 
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As a result of camera operation, a store displayed 
at the camera angle after the operation is determined as 
a new scope. 

The user can inform the map operating unit 105 of 
5 operation on the map 302 by clicking on a store area on the 
map 302. A store corresponding to the clicked area becomes 
a new scope. 

The displayed document 303 is a hypertext document. 
When the user clicks on a link portion (anchor) displayed 

10 with an underline, the current document is switched to 
another document. The store described by the new document 
becomes a new scope. 

The scope holding unit 101 holds a list of integral 
values (numbers) representing obj ects . In this embodiment , 

15 the scope holding unit 101 holds the following values : value 
1 representing XX restaurant, value 2 representing AA 
annex, value 3 representing □□ restaurant, value 4 
representingOO restaurant, value 5 representing Ox store, 
value 6 representing AD, value 7 representing xO 

20 restaurant, value 8 representing DO restaurant, value 9 
representing Ox center, value 10 representing AO 
restaurant, and value 11 representing AA restaurant. The 
scope holding unit 101 also holds value 0 which is a special 
value corresponding to no restaurant. When the scope 

25 holding unit 101 takes value 0, the picture display unit 
104 displays a default picture. In this case, the map 



display unit 107 displays a map without any camera angle, 
and a document display unit 110 displays a default document. 

Fig. 3 shows a display example displayed by each 
display unit when the scope holding unit 101 holds value 
5 2. A state wherein the camera angle of the camera installed 
at DO restaurant is set to AA annex is displayed on the 
map 302 . AA annex image-sensed by the camera is displayed 
on the picture 301. A hypertext document about AA annex 
is displayed on the document 303. 

10 When the user clicks on an at-a-glance button 322, 

a list of restaurants is displayed as anchors, as shown in 
Fig. 27. When the user clicks on an anchor corresponding 
to a desired restaurant, the scope held in the scope holding 
unit 101 is changed to the clicked object. 

15 A procedure for information display processing in 

this embodiment will be described below with reference to 
the flow chart of Fig. 4. Note that the processing shown 
in Fig. 4 is performed by the CPU 201. 

First of all, the CPU 201 performs initialization in 

20 step S401. The CPU 201 sets a predetermined value, e.g., 
0, in the scope holding unit 101. The flow then advances 
to step S402. 

In step S402, the picture switching unit 103 acquires 
a picture at a camera angle covering all objects 

25 corresponding to the held scope from the remote camera on 
the basis of the scope value held in the scope holding unit 



101, and switches the picture displayed by the picture 
display unit 104 to the acquired picture. The flow then 
advances to step S403. 

In step S403, the map switching unit 106 changes the 
5 graphically displayed state of the angle of a camera icon 
on the map 302 so as to cover all objects corresponding to 
the held scope in accordance with the scope value held in 
the scope holding unit 101. The flow then advances to step 
S404. 

10 In step S404, the document switching unit 109 

switches the document displayed by the document display 
unit 110 to a document describing an object corresponding 
to the held scope in accordance with the scope value held 
in the scope holding unit 101. The flow then advances to 

15 step S405. 

In step S405, the CPU 201 checks whether a camera 
operation input is received from the user at the picture 
operating unit 102. If YES in step S405, the flow advances 
to step S408. If NO in step S405, the flow advances to step 

20 S406. In step S406, the CPU 201 checks whether a map 
operation input is received from the user at the map 
operating unit 105. If YES in step S406, the flow advances 
to step S409. If NO in step S406, the flow advances to step 
S407. In step S407, the CPU 201 checks whether a document 

25 operation input is received from the user at the document 
operating unit 108. If YES in step S407, the flow advances 



to step S410. If NO in step S407, the flow returns to step 
S405. 

In step S408, the CPU 201 determines a camera angle 
in accordance with a picture operation input from the user. 
5 More specifically, a panning value, tilt value, and zoom 
value are determined for the camera in accordance with the 
positions of the slide bars 311, 312, and 313 moved by the 
user. The CPU 201 further obtains an object image-sensed 
at this camera angle, and sets a corresponding value in the 

10 scope holding unit 101 . If there are a plurality of obj ects 
image-sensed by the camera, a plurality of corresponding 
values are set in the scope holding unit 101. The flow then 
returns to step S402. 

In step S409, the CPU 201 obtains a newly selected 

15 object depending on the position on the map at which the 
user performs mouse clicking that corresponds to the map 
operation input from the user. An object in the area 
including the clicked point is selected. The CPU 201 then 
sets a value corresponding to the selected object in the 

20 scope holding unit 101. If a plurality of objects are 
selected by the user, values corresponding to all the 
objects are set in the scope holding unit 101. The flow 
then returns to step S4 02. 

In step S410, the CPU 201 obtains a document to be 

25 displayed next in accordance with the operation of tracing 
back a hyperlink, i.e., the document operation input from 



the user. The CPU 201 sets a value corresponding to the 
object described by the document in the scope holding unit 
101. The flow then returns to step S402. 

A method of obtaining an image-sensed object from a 
5 camera angle and a method of obtaining a camera angle 
covering all designated objects will be described next. 

Consider first a plane taking the panning and tilt 
values of the camera on the X-axis and Y-axis, respectively. 
When each object that can be image-sensed by the camera is 
10 image-sensed by the camera, each shape projected on this 
plane is considered as a polygon, and values corresponding 
to the respective vertexes of the polygon projected on the 
plane are generated and held in the picture switching unit 
103 in advance. 

15 When the camera angle is changed, the CPU 201 obtains 

intersections between a polygon (a sguare having sides 
obtained by subtracting/adding 1/2 the zoom value from/to 
the panning and tilt values) representing the current 
camera angle on the plane and the projected polygon of each 

20 object described above, thereby obtaining an object having 
the intersections as an object that is newly image-sensed. 

When the object (scope) to be image-sensed is changed, 
the CPU 201 calculates the sums of values representing the 
projected polygons of the respective objects, and obtains 

25 a square (having each side parallel to the X-axis or Y-axis) 
including all the sums. The center of gravity of this 
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square is set as the panning and tilt values of the camera, 
and the length of each side is set as the zoom value of the 
camera . 

In this embodiment, a table indicating the 
5 correspondence between the respective areas on the map and 
the objects and between the documents and the objects is 
generated in advance and stored in the disk unit 203. 
(Second Embodiment) 

In addition to the arrangement of the first 
10 embodiment, the second embodiment has an arrangement for 
presenting information as an advice from the information 
presenting apparatus side to a user. 

Fig. 5 is a block diagram showing the basic 
arrangement of the information presenting apparatus 
15 according to this embodiment. 

Referring to Fig. 5, reference numeral 111 denotes 
an advice generating unit for generating an advice about 
an object included in a scope in accordance with the scope 
held in a scope holding unit 101; and 112, an advice output 
20 unit for outputting the advice generated by the advice 
generating unit 111. 

Other blocks 101 to 109 have basically the same 
arrangements as those in the Fig. 1, and hence a description 
thereof will be omitted. 
25 Fig. 6 shows an example of information displayed on 

the display of the information presenting apparatus of this 



embodiment . 

Referring to Fig. 6, reference numerals 301 , 302, and 
303 respectively denote a picture, map, and document 
similar to those shown in Fig. 3; and 304, an advice output 
5 from the advice output unit 112. As this advice 304, 

information that is associated with an object included in 
the scope and likely to raise a user's interest is output. 
Assume that the object is a restaurant. In this case, if 
a restaurant having a menu that seems to match with a user' s 
10 taste is included in the scope, information indicating the 
presence of the menu and an explanation of the menu are 
displayed. 

If an object that cannot be image-sensed is included 
in the scope in the first and second embodiments, a default 
15 picture stored in advance may be displayed instead of a 
picture of the object. 

In the first and second embodiments, each scope 
corresponds to one object. However, the present invention 
is not limited to this. A scope may correspond to a 
20 plurality of objects. 

Fig. 24 shows a display example in the information 
presenting apparatus in such a case. If, for example, a 
plurality of objects (AA annex, □□ restaurant, and O 
O restaurant in Fig. 24) are selected on the map 302 in 
25 the display state shown in Fig. 3, the scope holding unit 
101 holds all the selected objects as a scope. The picture 
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switching unit 103 commands to control the camera to shift 
the zoom magnification to the wide-angle side so as to cover 
all the selected objects. At the same time, a document 
display unit 110 displays a list of documents about all the 
5 selected objects (referring to Fig. 24, in order to avoid 
complexity caused when pieces of information about the 
respective objects are displayed on the document 303 at once, 
only anchors 320 corresponding to the respective objects 
are displayed in this embodiment, and information 

10 corresponding to each object is presented through the 

network when the user clicks on a corresponding anchor) , 
as shown in Fig. 24. 

If, for example, the user operates a slide bar 313 
in the display state in Fig. 3 to shift the zoom 

15 magnification of the camera to the wide-angle side, all 
objects (AA annex, □□ restaurant, and OO restaurant in 
Fig. 24) existing in the image-sensing range of the camera 
are held as a scope in the scope holding unit 101. The 
document operating unit 108 then commands the map display 

20 unit 107 to switch the display of a camera symbol 314 on 
the map 302 to that shown in Fig. 24 so as to indicate that 
all the objects held in the scope holding unit 101 are 
image-sensed. At the same time, the document display unit 
110 displays a list of documents (the anchors 320 

25 corresponding to the respective objects are displayed in 
Fig. 24, and information is presented when the user clicks 



on the anchors) about all the objects held in the scope 
holding unit 101, as shown in Fig. 24. 

On the contrary, if the number of objects existing 
in the scope decreases, i.e., the state shown in Fig. 24 
5 shifts to the state shown in Fig. 3, the picture switching 
unit 103 controls the zoom magnification (also controls 
panning and tiJLting of the camera, as needed) of the camera 
to shift it to the telephoto side to image-sense the object 
held in the scope holding unit 101 as a main object, and 

10 the map display unit 107 is commanded to switch the display 
of the camera symbol 314 on the map 302 to that shown in 
Fig. 3. The document display unit 110 may display only a 
document about the object held in the scope holding unit 
101, as shown in Fig. 24. 

15 The picture/map/document switching operation 

described above is performed in steps S402 to S404 in 
Fig. 4. 

According to the above description, all objects as 
a scope can be image-sensed by one camera. If, however, 

20 all objects as a scope cannot be simultaneously 

image-sensed by one camera, a picture obtained by 
image-sensing only objects that can be simultaneously 
image-sensed may be displayed. A panoramic picture 
obtained by image-sensing objects at different camera 

25 angles and synthesizing the resultant pictures may be 

displayed, or a plurality of pictures obtained in advance 
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by image-sensing objects may be displayed. Alternatively, 
the camera angle may be automatically switched at 
predetermined intervals to image-sense the respective 
objects, and the resultant pictures may be displayed. 
5 If a desired object cannot be obtained by a camera 

as a control target even by displaying a panoramic picture 
or automatically operating the camera r a default picture 
stored in a disk unit 203 in advance may be displayed instead 
of the desired picture. 

10 If a list of stores is displayed as shown in Fig. 27, 

the scope held in the scope holding unit 101 includes all 
the stores presented in the list. The picture displayed 
by the picture display unit 301 is a default picture. 

Note that the advice generating unit 111 may generate 

15 information about each target as an advice. 

In addition, according to the above description, a 
single map is displayed by the map display unit 107. 
However, the present invention is not limited to this. A 
plurality of maps on different reduced scales may be held 

20 in the disk unit 203 and switched in accordance with the 
object to be image-sensed. In this case, the current map 
is switched to a map including all objects as a scope. For 
example, Fig. 25 shows such a case. 

Assume that the user has controlled the remote camera 

25 by operating slide bars 311 to 313. In this case, "A 

restaurant" as an object is included in the image-sensing 



range of the camera, and a scope corresponding to "A 
restaurant" is held in the scope holding unit 101 . However, 
"A restaurant" is not displayed on the map 302. The map 
switching unit 106 therefore performs control to switch the 
5 current map to a map on a reduced scale large enough to 
display the "A restaurant" as shown in Fig. 24. Note that 
data of objects to be image-sensed which can be displayed 
are added as attached information to a plurality of maps 
held in the disk unit 203. The map switching unit 106 

10 searches for a map on the basis of the attached information. 
In this case, of the maps that are searched out, a map on 
the minimum reduced scale may be displayed by the map 
display unit 107. 

Objects may be displayed by scrolling one map, as 

15 shown in Fig. 26, instead of changing the reduced scale of 
a map to be displayed as shown in Fig. 25. In this case, 
the map switching unit 10 6 holds information about each 
object displayed on the map 302, and checks on the basis 
of the information whether "A restaurant" is displayed. 

20 Upon determining that "A restaurant" is not displayed, the 
map switching unit 106 may command the map display unit 107 
to scroll the map to display "A restaurant". 

According to the third and fourth embodiments to be 
described below, the image-sensing condition of a camera 

25 is accurately detected to present information indicating 
whether a given object displayed on a map can be 



image-sensed by the camera or is actually image-sensed by 
the camera, in consideration of a case wherein an object 
designated by an instruction as a target to be image-sensed 
may differ from an object actually image-sensed by the 
5 camera because, for example, the object protrudes from the 
field defined by the field angle, resulting in confusion 
for the usi§r. 

Fig. 7 is a block diagram showing the basic 
arrangement of an information presenting apparatus 

10 according to the third embodiment. The practical 

arrangement of the third embodiment is the same as that 
shown in Fig. 2. 

Referring to Fig. 7, an image-sensing condition 
holding unit 113 holds the current image-sensing condition 

15 of a camera. The corresponding data is stored in a RAM 202 
in Fig. 2. The arrangement of the remaining portion is the 
same as that shown in Fig. 1 except that a picture switching 
unit 103 switches pictures by controlling the camera when 
a picture operating unit 102 changes the image-sensing 

20 range, the scope held in the scope holding unit 101 changes, 
or the image-sensing condition held in the image-sensing 
condition holding unit 113 changes, and a map switching unit 
106 changes map display when a map operating unit 105 
changes a scope, the scope held in the scope holding unit 

25 101 changes, or the image-sensing condition held in the 
image-sensing condition holding unit 113 changes. 



Although the arrangement of the remaining portion is the 
same as that shown in Fig. 1, the document operating unit, 
document switching unit, and document display unit are 
omitted from this embodiment. 
5 Fig. 8 shows an example of the data structure of the 

image-sensing condition holding unit 113. 

Referring to Fig. 8, a panning value 601 is expressed 
in degrees with the initial position of the camera being 
0, and a position in the right direction being a positive 

10 value. A tilt value 602 is expressed in degrees with the 
initial position of the camera being 0, and a position in 
the upward direction being a positive value. As a zoom 
value 603, the field angle of the camera in the horizontal 
direction is expressed in degrees. An integral value 604 

15 indicates whether all objects as a scope can be 

simultaneously image-sensed by the camera. If this value 
is 1, it indicates that all the objects can be image-sensed. 
If the value is 0, it indicates that the value of the current 
camera angle differs from the value held in the 

20 image-sensing condition holding unit 113, or not all the 
objects can be image-sensed at once. 

Fig. 9 shows an example of how information is 
displayed and presented by the display 204. 

Since components 301, 302, and 311 to 313 are the same 

25 as those in Fig. 3, a description thereof will be omitted. 
Reference numeral 314 denotes a highlighted portion 
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indicating the area of an object as a scope on the map 302. 
In addition, a camera angle is displayed on the map 302 by 
referring to the camera angle of the image-sensing 
condition information held in the image-sensing condition 
5 holding unit 113. The image-sensing condition holding 
unit 113 also displays the camera angle in blue on the map 
302 when the value 604 indicating whether all objects can 
be image-sensed is 1 (indicating that all objects can be 
image-sensed) , and displays the camera angle in red on the 

10 map 302 if the value 604 is 0 (indicating that not all objects 
can be image-sensed) . 

A procedure for information display processing in 
this embodiment will be described with reference to the flow 
chart of Fig. 10. 

15 First of all, initialization is performed in step 

S501. A predetermined value, e.g., 0, is set in the scope 
holding unit 101. The flow then advances to step S502. In 
step S502, the picture switching unit 103 determines a 
camera angle in accordance with the value of the scope held 

20 in the scope holding unit 101. More specifically, if there 
is a camera angle that covers all objects corresponding to 
the held scope, the picture switching unit 103 selects it. 
Otherwise, the picture switching unit 103 determines some 
appropriate angle. The picture switching unit 103 then 

25 acquire a picture at the determined camera angle from the 
remote camera, and switches the current picture on a picture 



display unit 104 to the acquired picture. In addition, the 
image-sensing condition of the camera is held in the 
image-sensing condition holding unit 113. The flow then 
advances to step S503. 
5 In step S503, the map switching unit 106 instructs 

a map display unit 107 to display a map covering all objects 
corresponding to the held scope in accordance with the value 
of the scope held in the scope holding unit 101, highlight 
an area (314 in Fig. 9) corresponding to the scope on the 

10 map in the color described above in accordance with the 
image-sensing condition held in the image-sensing 
condition holding unit 113, and display the camera angle 
on the map. The flow then advances to step S504. 

In step S504, it is checked whether a picture 

15 operation input is received from the user. If YES in step 
S504, the flow advances to step S506. If NO in step S504, 
the flow advances to step S505. In step S505, it is checked 
whether a map operation input is received from the user. 
If YES in step S505, the flow advances to step S507 . If 

20 NO in step S505, the flow returns to step S504 . 

In step S506, a new camera angle is determined in 
accordance with the picture operation input from the user. 
A panning value, tilt value, and zoom value are determined 
for the camera in accordance with the position of the camera 

25 after the slide bars 311, 312, and 313 are moved. These 
values and a value indicating that the scope can be 



image-sensed are held in the image-sensing condition 
holding unit 113. An object to be image-sensed at this 
camera angle is obtained, and a corresponding value is held 
in the scope holding unit 101. The flow then returns to 
5 step S503. 

In step S507, a newly selected object is obtained 
depending on the map operation input from the user, i.e., 
the position on the map at which the user has clicked. A 
value corresponding to the obtained object is held in the 

10 scope holding unit 101 . The flow then returns to step S502 . 

Note that the method of obtaining an image-sensed 
object from the camera angle and the method of obtaining 
a camera angle covering all designed objects are the same 
as those in the first embodiment. 

15 (Fourth Embodiment) 

In the third embodiment, a still image sensed most 
recently is used as a picture displayed on the picture 
display unit 104. However, the present invention is not 
limited to this. Moving images may be used. In this case, 

20 moving images are acquired from the camera and displayed 
concurrently with the processing described with reference 
to Fig. 10. When moving images are to be used, moving 
images at a designated camera angle may not be acquired if, 
for example, a plurality of users simultaneously use the 

2 5 camera. 

Such a problem may be solved by a method of displaying 
- 29 - 



a default picture or a method of storing the latest picture 
frame at a designed angle and outputting the corresponding 
picture as a still image. In this case, the displayed 
picture does not correspond to the current moving images 
5 at the camera angle displayed by a map display unit 107. 
In this embodiment, therefore, if the current camera angle 
does not correspond to the picture displayed on a picture 
display unit 104 because another user is operating the 
camera as in the above case, the camera angle is displayed 

10 in a different color (e.g., yellow). 

Fig. 11 is a block diagram showing an information 
presenting apparatus according to the fourth embodiment of 
the present invention. 

Referring to Fig. 11, reference numeral 114 denotes 

15 a camera condition acquiring unit 114 for acquiring, from 
a remote camera, information indicating whether moving 
images can be acquired at the camera angle designated by 
a picture switching unit 103 (i.e., whether the camera is 
currently used by another user) , and holds the information 

20 in an image-sensing condition holding unit 113. Since the 
arrangement of the remaining portion is the same as that 
shown in Fig. 7, a description thereof will be omitted. 
Fig. 12 shows an example of the data structure of the 
image-sensing condition holding unit 113 in this 

25 embodiment. 

Referring to Fig. 12, a panning value 701 is 
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expressed in degrees with the initial position of the camera 
being 0, and a position in the right direction being a 
positive value. A tilt value 702 is expressed in degrees 
with the initial position of the camera being 0, and a 
position in the upward direction being a positive value. 
As a zoom value 703, the field angle of the camera in the 
horizontal direction is expressed in degrees. An integral 
value 704 indicates whether all objects as a scope can be 
simultaneously image-sensed by the camera. If this value 
is 1, it indicates that all the objects can be image-sensed. 
If the value is 0, it indicates that not all the objects 
can be image-sensed at once. As a value 705, an integral 
value is held, which indicates whether any moving image has 
been acquired at the camera angle designated by the picture 
switching unit 103. If this value is 1, it indicates that 
a moving image at the designated camera angle has been 
acquired. If the value is 0, it indicates that no such an 
image has been acquired. 

Note that a procedure for information display 
processing in this embodiment is the same as that in the 
flow chart of Fig. 10 except for the processing in step 
S507. 

A camera angle 314 for which a map switching unit 106 
gives an instruction to display is displayed in the 
following colors. If the value 704 is 0, the camera angle 
is displayed in red. If the value 704 is 1, the display 
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color the camera angle changes in accordance with the value 
705. If the value 705 is 1, the display color is blue. If 
the value 705 is 0, the display color is yellow. Note that 
display color processing for the camera angle 314 is 
5 performed in step S507 in Fig. 10. 

In the third and fourth embodiments, for the sake of 
-descriptive convenience, scopes are switched by only 
picture operation and map operation. However, the present 
invention is not limited to this, and each embodiment may 
10 also include a document switching unit 109 as in the first 
and second embodiments. For example, a method of preparing 
a list of objects and selecting an object as a scope from 
the list is available. 

(Fifth, Sixth, and Seventh Embodiments) 

15 There is provided a camera controller designed to 

control a camera with a camera angle at which an arbitrary 
object designated on a map is to be image-sensed and perform 
camera control to acquire information identifying an object 
image-sensed at an arbitrary camera angle. 

20 Fig. 13 is a block diagram showing the basic 

arrangement of a camera controller according to the fifth 
embodiment of the present invention. 

Referring to Fig. 13, reference numeral 801 denotes 
a conversion parameter holding unit for holding parameters 

25 used for conversion between the coordinate system of the 
camera and the coordinate system of a map; 802, an area data 
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holding unit for holding data describing the area occupied 
by an object on a map in the coordinate system of the map; 
803, a camera angle holding unit for holding the angle data 
of the camera; and 804, a scope holding unit for holding 
5 a list of scopes which are objects currently image-sensed 
by the camera. 

Reference numeral 8-05 denotes a scope request holding 
unit for holding a list of objects that are newly requested 
to be image-sensed by the camera; 806, an angle computing 
10 unit for obtaining a camera angle at which the objects held 
in the scope request holding unit 805 are simultaneously 
image-sensed, by using the data in the conversion parameter 
holding unit 801 and area data holding unit 802, and holding 
the new angle in the camera angle holding unit 803; and 807, 
15 a scope computing unit for obtaining an object image-sensed 
at the camera angle held in the camera angle holding unit 
803 by using the data in the conversion parameter holding 
unit 801 and area data holding unit 802, and holding the 
object as a scope in the scope holding unit 804. 
20 Fig. 14 shows a practical arrangement of the camera 

controller of this embodiment. 

Referring to Fig. 14, reference numeral 904 denotes 
a CPU that operates in accordance with a program for 
implementing a procedure to be described later; 902, a RAM 
25 for providing the camera angle holding unit 803, the scope 
holding unit 804, the scope request holding unit 805, and 



a storage area required for operation of the program; 903, 
a disk unit for holding the conversion parameter holding 
unit 801, the area data holding unit 802, and a program for 
implementing a procedure to be described later; and 904, 
5 a bus. 

Fig. 15 shows an example of a map on which area data 
is based. Buildings on this map are set as objects. 
Integral values (numbers) set for the respective areas, 
e.g., the area of AA annex and the like are identical to 

10 the values 1 to 13 in Fig. 3 in the first embodiment. 

Fig. 16 shows an example of the data structure of the 
area data holding unit 802. 

Referring to Fig. 16, each row corresponds to one 
objet. A column 501 is an area for holding the number of 

15 an object which a corresponding row describes. Column 502 
is an area for holding a list of combinations of X- and 
Y-coordinates of the vertexes of a polygon representing the 
area of an object which a corresponding row describes. In 
the map coordinate system, the upper left corner of the map 

20 is set as the origin, the X-axis is set in the horizontal 
direction with the right direction being a positive 
direction, and the Y-axis is set in the vertical direction 
with the downward direction being a positive direction. 

A real space expressed on a map is expressed in a 

25 coordinate system having the X-axis and Y-axis expressed 
in the same coordinates as those of the map and the Z-axis 
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set in the direction perpendicular to the origin of the map 
with the upward direction being a positive direction. 

In the camera coordinate system, the imaging point 
of the camera is set as the origin, an axis along which the 
5 image-sensing direction of the image-sensing axis becomes 
a positive direction when the panning and tilt values of 
the camera are 0 is set as the Z-axis, a line intersecting 
a panning plane in a plane that passes through the imaging 
point and is perpendicular to the Z-axis is set as the X-axis 

10 (with the right direction being a positive direction) , and 
a line intersecting a tilt plane is set as the Y-axis (with 
the upward direction being a positive direction) . Assume 
that the X-axis of the camera coordinate system is always 
parallel to a horizontal plane. That is, the camera is not 

15 tilted in the horizontal direction. 

Let a be the angle defined by a line obtained by 
projecting the Z-axis of the camera coordinate system on 
the X-Y plane of the map coordinate system and the Y-axis 
of the map coordinate system, and j3 be the angle defined 

20 by a line along which the Y-Z plane of the camera coordinate 
system intersects a horizontal plane and the Z-axis of the 
camera coordinate system. In addition, the coordinate 
value of the installation position of the camera in the map 
coordinate system is represented by (XC, KC, ZC) . 

25 Fig. 17 shows an example of the data structure of the 

conversion parameter holding unit 801. 
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Referring to Fig. 17, reference numeral 401 denotes 
an area for holding the value a; 402, an area for holding 
the value j8 ; 403, an area for holding the value XC; 404, 
an area for holding the value YC; and 405, an area for holding 
5 the value ZC. 

Reference numeral 406 denotes an area for holding a 
value PI expressing the panning limit value of the camera 
in the left direction in degrees; 407, an area for holding 
a value Pr expressing the panning limit value of the camera 

10 in the right direction in degrees; 408, an area for holding 
a value Td expressing the tilting limit value of the camera 
in the downward direction in degrees; 409, an area for 
holding a value TU expressing the tilting limit value of 
the camera in the upward direction in degrees; 410, an area 

15 for holding a value gt expressing the zooming limit value 
of the camera on the telephoto side in degrees of field angle 
in the panning direction; 411, an area for holding a value 
qw expressing the zooming limit value of the camera on the 
wide-angle side in degrees of field angle in the panning 

20 direction; and 412, an area for holding a ratio r of the 
field angle of the camera in the panning direction to the 
field angle in the tilting direction. 

The values in the areas 401 to 405 are set when this 
apparatus is constructed. The values in the areas 406 to 

25 412 are acquired in advance from a camera to be used. 

Fig. 18 shows an example of the data structure of the 



camera angle holding unit 803. 

Referring to Fig. 18, reference numeral 1201 denotes 
an area for holding a panning value p of the camera which 
is expressed in degrees with the initial position of the 
5 camera being 0, and a position in the right direction being 
a positive value; 1202, an area for holding a tilt value 
t of the camera which is expressed in degrees with the 
initial position of the camera being 0, and a position in 
the upward direction being a positive value; and 1203, an 
10 area for holding a zoom value q of the camera which is 
expressed as the field angle of the camera in degrees. 

A procedure for processing in this embodiment will 
be described next with reference to the flow chart of 
Fig. 19. 

15 First of all, initialization is performed in step 

S301. The scope holding unit 804 is emptied, and 
predetermined values are set in the camera angle holding 
unit 803. Examples of the predetermined values are p = 0, 
t = 0, and q = 30. The flow then advances to step S302. 

20 In step S302, it is checked whether the angle value 

held in the camera angle holding unit 803 is changed. If 
YES in step S302, the flow advances to step S303. If NO 
in step S302, the flow advances to step S305. If, however, 
the angle value is changed by the scope computing unit 807, 

25 the same operation as that performed when the angle value 
is not changed is performed. In step S303, on the basis 



- 37 - 



of the new camera angle value, a scope as a list of objects 
image-sensed at the angle is computed. This calculation 
method will be described in detail later. The flow then 
advances to step S304. 
5 In step S304, the scope value obtained in step S303 

is held in the scope holding unit 804. The flow returns 
to step S302. In step S305, it is checked whether a scope 
request is held in the scope request holding unit 805. If 
YES in step S305, the flow advances to step S306. If NO 

10 in step S305, the flow returns to step S302. In step S306, 
a camera angle, of camera angles at which all the objects 
listed as objects corresponding to the scope request can 
be image-sensed, at which the zooming value is closest to 
the telephoto end is obtained, and the contents of the scope 

15 request holding unit 805 are erased. This calculation 
method will be described in detail later. The flow then 
advances to step S307 . 

In step S307, is it checked whether an appropriate 
angle is obtained in step S306. If YES in step S307, the 

20 flow advances to step S308. If NO in step S307, the flow 
returns to step S302. In step S308, the camera angle 
obtained in step S306 is held in the camera angle holding 
unit 8 03, and the value of the scope request used for the 
computation is held in the scope holding unit 804 . The flow 

25 then returns to step S302. 

The method of obtaining an image-sensed object from 
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the camera angle will be described in detail next. 

Consider each object On (n represents the number of 
each object) . A side of the area of a given object that 
can be viewed from the camera without being hidden behind 
another object is extracted. In a vertical plane including 
this side, a rectangle formed by an area with a 
predetermined height from the side is represented by Sn. 
Let Rn be a rectangle formed with the same topology as that 
of the original rectangle by using points at which straight 
lines connecting the respective vertexes of the rectangle 
Sn to the origin of the camera coordinate system intersect 
a plane expressed by z = 1 in the camera coordinate system. 
That is, if a vertex of the rectangle Sn is expressed by 
(xs, ys, zs) , a corresponding vertex of the rectangle Rn 
is expressed by (xs/zs, ys/zs, 1). 

Although a plurality of such rectangles may be set 
for one object, one rectangle is set for each object in this 
case for the sake of simplicity. This assumption holds in 
the case shown in Figs. 15 and 17. Assume that a plurality 
of rectangles are set . Even in this case, no problem arises 
if a graphic pattern obtained by overlaying the rectangles 
is handled as the rectangle Rn in the following processing. 

In this case, the coordinate value of a rectangle must 
be converted from the map coordinate system to the camera 
coordinate system. This conversion is performed by using 
the values in the conversion parameter holding unit 801 as 



follows. If a given point is expressed by (xm, ym, zm) in 
the map coordinate system, the expression (xv, yv, zv) of 
the same point in the camera coordinate system is obtained 
by: 

xv = cosa (xm - Xc) + sinCK(zm - Zc) 

yv = sinasin/3 (xm - Xc) + cos j3 (ym - Yc) - cos a 
- sin /3 ( zm - Zc) 

zv = sina:cosj3 (xm - Xc) - sin/3 (ym - Yc) - cosa 
cos B (zm - Zc) ... (1) 

Then, a rectangle A obtained by projecting the area 
image-sensed by the camera on a plane expressed by z = 1 
is obtained. The rectangle A is an area enclosed with the 
following four straight lines: 

x = tan(p - q/2) 

x = tan (p + q/2) 

x = tan(t - rq/2) 

x = tan (t + rq/2) ... (2) 

With regards to all the objects, it is checked whether 
the rectangles Sn and A intersect each other. If the 
rectangles Sn and A intersect, the corresponding object is 
included in the scope. 

With the above processing, a scope can be obtained 
from the camera angle. 

The method of obtaining a camera angle from a scope 
request will be described in detail next. 

Letting B be a graphic pattern obtained by overlaying 



- 40 - 



the rectangles Rn of all objects included in s scope request, 
and xrain, xmax, ymin, and ymax be the maximum and minimum 
values of the X- and Y-coordinates of a point in the graphic 
pattern B, a camera angle is determined as follows: 

p = (arctan xmin + arctan xmax) /2 

t = (arctan ymin + arctan ymax) /2 

q = max [ (arctan xmin - arctan xmax) , { (arctan ymax 

- arctan ymin)/r}] ...(3) 
where max is a function taking the maximum of an argument. 

If pi ^ p ^ pr and td ^ t ^ tu and qt ^ q ^ qw, the 
above value is set as a new camera angle. 

In the fifth embodiment, all conversion parameters 
are provided in advance. However, the present invention 
is not limited to this, and some of the parameters may be 
generated. 

Fig. 20 is a block diagram showing the basic 
arrangement of a camera controller for obtaining the above 
values a , (3 , and Zc by using image-sensing sampling points 
according to the sixth embodiment of the present invention. 

Referring to Fig. 20, reference numeral 808 denotes 
an image-sensing sampling point holding unit for holding 
image-sensing sampling points; and 809, a conversion 
parameter computing unit for obtaining some conversion 
parameters from the image-sensing sampling points held in 
the image-sensing sampling point holding unit 808 and other 
parameters held in the conversion parameter holding unit 



801. 

The arrangement of the remaining portion is the same 
as that shown in Fig. 13. 

The process of obtaining conversion parameters from 
5 image-sensing sampling points will be described below. 
Other processes are the same as in the case shown in Fig. 13 . 

An image-sensing sampling point is a combination of 
a panning value, a tilt value, and the X- and Y-coordinates 
of the image-sensing position which are set when 
10 image-sensing is performed by a camera such that a given 
position on the surface of the earth coincides with the 
center of a picture taken by the camera. Three such points 
are prepared, and each point is represented by (Pn, Tn, Xn, 
Yn) (n = 0, 1, 2) . 
15 Even if all the value a , j3 , and Zc are unknown, these 

values can be obtained from the above image-sensing 
sampling points as follows: 

a = arctanf (-b +^(b2 - 4ac) /2a} - --(4) 

for -90 ^ a ^ 90 
20 In addition, a, b, and c are given as follows: 

a = (X0 - X2) (Zl tanPO - Z0 tanPl)tanP2 - (X0 - XI) (Z2 

tanPO - Z0 tanP2)tanPl 
b = (X0 - X2) (XI tanPO - X0 tanPl)tanP2 + (Z2 - Z0) (Zl 
tanPO - Z0 tanPl)tanP2 - (X0 - XI) (X2 tanPO - X0 
25 tanP2)tanPl - (Zl - Z0) (Z2 tanPO - Z0 tanP2)tanPl 

c = (Z2 - Z0) (XI tanPO - X0 tanPl)tanP2 - (Zl - Z0) (X2 
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tanPO - XO tanP2)tanP2 ... (5) 

£ = arcsin[{XO tanTO tanPl - XI tanTl tanPO}cosa + 
{ZO tanTO tanPl - Zl tanTl tanPO } sin a } ] / [ { (XO 
- Xl)sinff + (Zl - ZO) cos a } tanPO tanPl}] 
Zc = (XO sina - ZO cosa)tan/3 - [{ (XO cos a + ZO sin 
& ) tanTO } tanTO /tanPO cos ]3 ] ...(6) 
- The seventh embodiment of the present invention will 
be described next. 

In the fifth and sixth embodiments, the values a , 
j3 , and Zc are obtained from three image-sensing sampling 
points. However, the present invention is not limited to 
this. Another combination of input and output values that 
can be calculated may be calculated. For example, values 
Xc and Yc can be further obtained from four sampling points . 

According to the above description, another 
apparatus sets and uses an angle to be held in the angle 
holding unit 803 and a scope to be held in the scope holding 
unit 805. However, the present invention is not limited 
to this, and these operations may be performed within this 
apparatus . 

Fig. 21 is a block diagram showing the basic 
arrangement of a camera controller according to the seventh 
embodiment like the one described above. 

Referring to Fig. 21, reference numerals 801 and 807 
denote units identical to those in Fig. 13; 810, a camera 
operating unit for receiving information representing 



camera angle changing operation and holding a new angle in 
an angle holding unit 803; 811, a camera interface unit for 
notifying the camera of an instruction to set it to the 
camera angle held in the angle holding unit 803; 812, a 
5 picture display unit for displaying a picture acquired 
through the camera interface unit; 813, a map display unit 
for displaying a map and highlighting the area of an object 
belonging to the scope held in a scope holding unit 804; 
and 814, a map operating unit for receiving information 
10 representing object selecting operation on the map and 
holding a list of selected objects in a scope request 
holding unit 8 05. 

Fig. 22 shows a practical arrangement of the camera 
controller of this embodiment. 
15 Referring to Fig. 22, reference numeral 1001 denotes 

a CPU that operates in accordance with a program for 
implementing a procedure to be described later; 1002, a RAM 
for providing the angle holding unit 803, the scope holding 
unit 804, the scope request holding unit 805, and a storage 
20 area required for operation based on the above program; 1003, 
a disk unit for holding the conversion parameter holding 
unit 801, the area data holding unit 802, and a program for 
implementing a procedure to be described later; 1004, a 
display for output display information from the picture 
25 display unit 812 and map display unit 813; 1005, a mouse 
that is used to input operation commands to the camera 



operating unit 810 and map operating unit 814; 1006, a 
network interface for implementing communication between 
the camera interface unit 811 and the camera; and 1007, a 
bus . 

5 Fig. 23 shows an example of information displayed on 

the display of the camera controller according to this 
embodiment . 

Referring to Fig. 23, reference numeral 1101 denotes 
an input interface for receiving pictures output from the 

10 picture display unit 812 and inputting information to the 
camera operating unit 810. When the user moves slide bars 
1111, 1112, and 1113 above the input interface 1101, 
information representing picture operation can be 
transferred to the camera operating unit 810. The user can 

15 laterally move the camera by laterally moving the slide bar 
1111 . The user can move the camera vertically by vertically 
moving the camera by vertically moving the bar 1112. The 
user can change the zoom value of the camera by vertically 
moving the slide bar 1113. 

2 0 Reference numeral 1102 denotes a map output from the 

map display unit 813. All objects as a scope held in the 
scope holding unit 804 are displayed on the map, and the 
area of an object of the scope is indicated by a highlighted 
portion 1114 on the map. When the user clicks on the area 

25 of a store on the map 1102, information representing this 
map operation can be transferred to the map operating unit 



813. The number (integral value) of the store 
corresponding to the clicked area is held in the scope 
holding unit 805. 

In the above embodiments, angle computation is 
5 performed to obtain a camera angle covering all buildings. 
However, the present invention is not limited to this. An 
angle covering part of a building may be obtained^ In, for 
example, obtaining a camera angle, the height of a building 
may be set to smaller than the value used for scope 

10 computation or the measured value, and an angle at which 
only a portion near the surface of the earth can be 
image-sensed may be obtained. 

According to the above description, scopes are 
switched by performing only picture operation and map 

15 operation. However, the present invention is not limited 
to this. As in the first embodiment, this apparatus may 
also include a document switching means. For example, a 
list of objects is prepared, and an object as a scope is 
selected from this list, and information about the object 

20 belonging to the scope is displayed. 

A storage medium according to this embodiment will 
be described next. 

The first to seventh embodiments shown in the 
accompanying drawings can be implemented by hardware 

25 circuits. When each embodiment is to be implemented by a 
computer system constituted by a CPU and a memory such as 



a disk unit, the memory constitutes the storage medium of 
the present invention. This storage medium stores 
programs for executing the processing and operation 
described above in each embodiment with reference to the 
flow charts and the like. 

As this storage medium, for example, a ROM, 
semiconductor memory such as a RAM, optical disk, 
magneto-optical disk, or a magnetic recording medium may be 
used. These media may be formed into a CD-ROM, FD, 
magnetic card, magnetic tape, nonvolatile memory card, and 
the like. 

The same functions and effects as those of each 
embodiment described above can be obtained, and the objects 
of the present invention can be achieved by using this storage 
medium in a system or apparatus other than the systems shown 
in the accompanying drawings, and causing the system or 
computer to read out and execute the program codes stored 
in the storage medium. 

The same functions and effects as those of each 
embodiment described above can be obtained, and the objects 
of the present invention can be achieved when an OS running 
on the computer executes some or all of processing 
operations, or when a CPU or the like arranged in a function 
extension board or a function extension unit executes some 
or all of processing operations on the basis of instructions 
of the program codes read out from the storage medium after 



the program codes are written in a memory of the extension 
board or unit inserted/connected in/to the computer. 

As has been described above, according to each 
embodiment described above, information about an object 
5 image-sensed by the camera can be automatically presented, 
and an object described by information browsed by the user 
can be automatic-ally image-sensed by the camera and 
presented while the use is browsing the information. The 
user can therefore easily acquire information while seeing 
10 the current picture of an object, or can easily acquire a 
picture while browsing information, without performing any 
specific operation. 

In addition, according to each embodiment described 
above, an object for which an instruction to image-sense is 
15 given and an angle at which the object is actually 

image-sensed are presented in a readable form, and the 
image-sensing condition of the camera can also be presented. 

Furthermore, according to this embodiment, when an 
arbitrary object on a map is designated, the angle of a camera 
20 for image-sensing the object can be controlled. When an 
arbitrary camera angle is set, information identifying an 
object image-sensed at the arbitrary angle can be obtained. 

As many apparently widely different embodiments of 
the present invention can be made without departing from 
25 the spirit and scope thereof, it is to be understood that 
the invention is not limited to the specific embodiments 



thereof except as defined in the appended claims. 



WHAT IS CLAIMED IS: 

1. An information presenting system comprising: 

scope holding means for holding a scope indicating 
an object to which a user is currently giving attention in 
5 accordance with at least one of a picture sensed by a camera, 
a map indicating a position of the camera, and a text 
document ; 

picture operating means for changing an 
image-sensing range of the camera; 
10 first instruction means for instructing the camera 

to switch display pictures when the image-sensing range of 
the camera is changed by said camera operating means or the 
scope held in said scope holding means is changed, and 
instructing said scope holding means to hold a new scope 
15 corresponding to a new image-sensing range when the 

image-sensing range of the camera is changed by said picture 
display means; 

picture display means for displaying a picture within 
the image-sensing range of the camera which is designated 
20 by an instruction from said first instruction means; 

map operating means for changing the scope in said 
scope holding means by designating an object on the map by 
giving an instruction; 

second instruction means for updating display of a 
25 symbol indicating the image-sensing range of the camera 
which is displayed on the map when the scope is changed by 
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said map operating means or the scope held in said scope 
holding means is changed, and instructing said scope 
holding means to hold a new scope when the scope is changed 
by said map operating means; 
5 map display means for displaying a map designated by 

an instruction from said second instruction means; 

docume-nt operating means for changing a display 
document by operating a text document ; 

third instruction means for giving an instruction to 

10 switch documents when the display document is changed by 
said document operating means or the scope held in said 
scope holding means is changed, and instructing said scope 
holding means to hold a new scope when the display document 
is changed by said document operating means; and 

15 document display means for displaying the document 

designated by an instruction from said third instruction 
means . 

2. The system according to claim 1, wherein if the scope 
held in said scope holding means is changed by said first 

20 or third instruction means, and an object corresponding to 
a new scope is not displayed on said map display means, said 
map display means is instructed to display the object. 

3. The system according to claim 2, wherein said system 
further comprises storage means for storing maps on 

25 different reduced scales, and if the scope held in said 
scope holding means is changed by said first or third 
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instruction means, and an object corresponding to a new 
scope is not displayed on said map display means, said 
second instruction means instructs said map display means 
to display a map on the smallest reduced scale that can 
5 display a corresponding new image-sensing range. 

4 . The system according to claim 2, wherein if the scope 
held in said scope holding means is changed by said first 
or third instruction means, and an object corresponding to 
a new scope is not displayed on said map display means, said 
10 second instruction means instructs said map display means 
to scroll the displayed map so as to display a corresponding 
new image-sensing range. 

5. The system according to claim 1, wherein if there are 
a plurality of scopes to be held in said scope holding means, 

15 said first instruction means designates control on the 
camera to image-sense all objects corresponding to the 
scopes . 

6. The system according to claim 5, wherein if the camera 
cannot simultaneously image-sense the plurality of objects 

20 corresponding to the scopes held in said scope holding means , 
said first instruction means designates control on the 
camera to image-sense each object at predetermined time 
intervals . 

7 . The system according to claim 5, wherein if the camera 
25 cannot simultaneously image-sense the plurality of objects 
corresponding to the scopes held in said scope holding means, 
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said first instruction means displays a panoramic image 
obtained by image-sensing the respective objects. 
8 . The system according to claim 1, wherein if the camera 
cannot image-sense the object corresponding to the scope 
5 held in said scope holding means, said first instruction 
means displays a picture taken in advance. 
9. The system according to claim 1, wherein if one scope 
is held in said scope holding means, said third instruction 
means instructs said document display means to display a 

10 text document having information about an object 

corresponding to the scope, and if a plurality of scopes 
are held in said scope holding means, said third instruction 
means instructs said document display means to display only 
anchors corresponding to the scopes. 

15 10. The system according to claim 1, wherein said system 
further comprises acquiring means for acquiring 
information about a current image-sensing range of the 
camera, and said second instruction means changes a display 
mode of a symbol indicating the image-sensing range of the 

20 camera depending on whether the information about the 

current image-sensing range of the camera coincides with 
the image-sensing range of the camera which is designated 
by an instruction from said first instruction means. 
11. An information presenting method comprising: 

25 the scope holding step of holding a scope indicating 

an object to which a user is currently giving attention in 
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accordance with at least one of a picture sensed by a camera, 
a map indicating a position of the camera, and a text 
document ; 

the picture operating step of changing an 
5 image-sensing range of the camera; 

the first instruction step of instructing the camera 
to switch display pictures when the image-sensing range of 
the camera is changed in the camera operating step or the 
scope held in the scope holding step is changed, and giving 
10 an instruction to hold a new scope corresponding to a new 
image-sensing range in the scope holding step when the 
image-sensing range of the camera is changed in the picture 
display step; 

the picture display step of displaying a picture 
15 within the image-sensing range of the camera which is 

designated by an instruction in the first instruction step; 

the map operating step of changing the scope in the 
scope holding step by designating an object on the map by 
giving an instruction; 
20 the second instruction step of updating display of 

a symbol indicating the image-sensing range of the camera 
which is displayed on the map when the scope is changed in 
the map operating step or the scope held in the scope holding 
step is changed, and giving an instruction to hold a new 
25 scope in the scope holding step when the scope is changed 
in the map operating step; 
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the map display step of displaying a map designated 
by an instruction from in the second instruction step; 

the document operating step of changing a display 
document by operating a text document; 
5 the third instruction step of giving an instruction 

to switch documents when the display document is changed 
in the document operating step or the scope held in the scope 
holding step is changed, and giving an instruction to hold 
a new scope in the scope holding step when the display 

10 document is changed in the document operating step; and 
the document display step of displaying the document 
designated by an instruction in the third instruction step. 
12 . The method according to claim 11, wherein if the scope 
held in the scope holding step is changed by the first or 

15 third instruction step, and an object corresponding to a 
new scope is not displayed on the map display step, the map 
display step comprises instructing to display the object. 
13. The method according to claim 12, wherein the method 
further comprises the storage step of storing maps on 

20 different reduced scales, and the second instruction step 
comprises, if the scope held in the scope holding step is 
changed in the first or third instruction step and an object 
corresponding to a new scope is not displayed in the map 
display step, giving an instruction to display a map on the 

25 smallest reduced scale that can display a corresponding new 
image-sensing range in the map display step. 
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14. The method according to claim 12, wherein the second 
instruction step comprises, if the scope held in the scope 
holding step is changed in the first or third instruction 
step and an object corresponding to a new scope is not 

5 displayed in the map display step, giving an instruction 
to scroll the displayed map so as to display a corresponding 
new image-sensing range- in the map display step. 

15. The method according to claim 11, wherein the first 
instruction step comprises, if there are a plurality of 

10 scopes to be held in the scope holding step, instructing 
control on the camera to image-sense all objects 
corresponding to the scopes. 

16. The method according to claim 15, wherein the first 
instruction step comprises, if the camera cannot 

15 simultaneously image-sense the plurality of objects 

corresponding to the scopes held in the scope holding step, 
instructing control on the camera to image-sense each 
object at predetermined time intervals. 

17. The method according to claim 15, wherein the first 
20 instruction step comprises, if the camera cannot 

simultaneously image-sense the plurality of objects 
corresponding to the scopes held in the scope holding step, 
displaying a panoramic image obtained by image-sensing the 
respective objects. 
25 18. The method according to claim 11, wherein the first 
instruction step comprises, if the camera cannot 
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image-sense the object corresponding to the scope held in 
the scope holding step, displaying a picture taken in 
advance . 

19. The method according to claim 11, wherein the third 
5 instruction step comprises, if one scope is held in the 

scope holding step, giving an instruction to display a text 
document having information about an object corre-sponding 
to the scope in the document display step, and if a plurality 
of scopes are held in the scope holding step, giving an 
10 instructing to display only anchors corresponding to the 
scopes in the document display step. 

20. The method according to claim 11, wherein the method 
further comprises the acguiring step of acquiring 
information about a current image-sensing range of the 

15 camera, and the second instruction step comprises changing 
a display mode of a symbol indicating the image-sensing 
range of the camera depending on whether the information 
about the current image-sensing range of the camera 
coincides with the image-sensing range of the camera which 

20 is designated by an instruction in the first instruction 
step. 

21. A computer-readable medium storing a program 
comprising program codes of: 

the scope holding step of holding a scope indicating 
25 an object to which a user is currently giving attention in 
accordance with at least one of a picture sensed by a camera, 
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a map indicating a position of the camera, and a text 
document ; 

the picture operating step of changing an 
image-sensing range of the camera ; 
5 the first instruction step of instructing the camera 

to switch display pictures when the image-sensing range of 
the camera is changed in the camera operating step or the 
scope held in the scope holding step is changed, and giving 
an instruction to hold a new scope corresponding to a new 
10 image-sensing range in the scope holding step when the 
image-sensing range of the camera is changed in the picture 
display step; 

the picture display step of displaying a picture 
within the image-sensing range of the camera which is 
15 designated by an instruction in the first instruction step; 

the map operating step of changing the scope in the 
scope holding step by instructing an object on the map by 
giving an instruction; 

the second instruction step of updating display of 
20 a symbol indicating the image-sensing range of the camera 
which is displayed on the map when the scope is changed in 
the map operating step or the scope held in the scope holding 
step is changed, and giving an instruction to hold a new 
scope in the scope holding step when the scope is changed 
25 in the map operating step; 

the map display step of displaying a map designated 
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by an instruction in the second instruction step; 

the document operating step of changing a display- 
document by operating a text document; 

the third instruction step of giving an instruction 
5 to switch documents when the display document is changed 
in the document operating step or the scope held in the scope 
holding step is changed, and giving an instruction to hold 
a new scope in the scope holding step when the display 
document is changed in the document operating step; and 

10 the document display step of displaying the document 

designated by an instruction in the third instruction step. 
22 . The medium according to claim 21, wherein if the scope 
held in the scope holding step is changed by the first or 
third instruction step, and an object corresponding to a 

15 new scope is not displayed on the map display step, the map 
display step comprises instructing to display the object. 
23. The medium according to claim 22, wherein the medium 
further comprises a program code of the storage step of 
storing maps on different reduced scales, and the second 

20 instruction step comprises, if the scope held in the scope 
holding step is changed in the first or third instruction 
step and an object corresponding to a new scope is not 
displayed in the map display step, giving an instruction 
to display a map on the smallest reduced scale that can 

25 display a corresponding new image-sensing range in the map 
display step. 



24. The medium according to claim 22, wherein the second 
instruction step comprises, if the scope held in the scope 
holding step is changed in the first or third instruction 
step and an object corresponding to a new scope is not 

5 displayed in the map display step, giving an instruction 
to scroll the displayed map so as to display a corresponding 
new image-sensing range in the map display step. 

25. The medium according to claim 21, wherein the first 
instruction step comprises, if there are a plurality of 

10 scopes to be held in the scope holding step, instructing 
control on the camera to image-sense all objects 
corresponding to the scopes. 

26. The medium according to claim 25, wherein the first 
instruction step comprises, if the camera cannot 

15 simultaneously image-sense the plurality of objects 

corresponding to the scopes held in the scope holding step, 
instructing control on the camera to image-sense each 
object at predetermined time intervals. 

27. The medium according to claim 25, wherein the first 
20 instruction step comprises, if the camera cannot 

simultaneously image-sense the plurality of objects 
corresponding to the scopes held in the scope holding step, 
displaying a panoramic image obtained by image-sensing the 
respective objects. 
25 28. The medium according to claim 21, wherein the first 
instruction step comprises, if the camera cannot 
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image-sense the object corresponding to the scope held in 
the scope holding step, displaying a picture taken in 
advance . 

29. The medium according to claim 21, wherein the third 
5 instruction step comprises, if one scope is held in the 

scope holding step, giving an instruction to display a text 
document having information about an object corresponding 
to the scope in the document display step, and if a plurality 
of scopes are held in the scope holding step, giving an 
10 instructing to display only anchors corresponding to the 
scopes in the document display step. 

30. The medium according to claim 21, wherein the medium 
further comprises a program code of the acquiring step of 
acquiring information about a current image-sensing range 

15 of the camera, and the second instruction step comprises 
changing a display mode of a symbol indicating the 
image-sensing range of the camera depending on whether the 
information about the current image-sensing range of the 
camera coincides with the image-sensing range of the camera 

20 which is designated by an instruction in the first 
instruction step. 
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Abstract of the Disclosure 
The present invention has as its object to 
automatically provide information about an object 
image-sensed by a camera and automatically image-sense and 
5 present an object described by browsed information while 
a user is browsing the information. To achieve this object, 
the present invention comprises scope holding means, 
picture operating means, first instruction means, picture 
display means, map operating means, second instruction 
10 means, map display means, document operating means, third 
instruction means and document display means. 
31. 
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